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WHAT IS CLAIMED IS: 

1 1 . In a content delivery system having m servers, S ' = {S x , . . . ,5 m } , n active 

2 customers , C = { Q , . . . , C n } , and g geographic locations , G ' = {G u ..'. 9 G g } 9 wherein sdel k is a 

3 server delay of server S k9 ndel j k is a network delay observed by customers in geographic 

4 location G ; while retrieving content from server S k9 Pj is a priority value for customer C i9 q is a 

5 total load of customer Q', u itj is a fraction of requests coming to customer C, from region G J9 
_6 a ( - - fJk is a mapping representing a fraction of requests coming to customer C, from region G, that 
y37 have been redirected to server S k9 and s k represents a load capacity of server S k9 a method for 
ri8 distributing server loads, the method comprising the steps of: 

representing an average prioritized observed response time as 

y n — , g — , m 

- L._ 1 Z,._ 1 2. Zr _i a i.M X U U X c i X Pi x ( sdel * + ndel U ) 

40 AORT= l ~ l J ~ [ fC ~ l ;and 



, = 1 c,xp, 



u 

yj 

Nt 1 generating a mapping that assigns requests from customers to a particular 

£12 server while minimizing AORT. 

1 2. A method as recited in claim 1, further including the step of assigning all 

2 requests from all customers in all regions to a particular server such that, for each 



3 C t eC\G t eG' t £ fl^=1.0. 

A=1 



1 3. A method as recited in claim 1, further including the step of assigning 

2 requests to a particular server while ensuring that the load capacity of each server is not 

n g 

3 exceeded such that, for each for each S k e S\ J] ^ a i Jk x u i} x c i < s k . 

7=1 
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1 4. A method as recited in claim 1, further including the step of assigning 

2 requests to a particular server while balancing the load of each server to within a maximum 

3 allowed deviation from a balanced state © such that, for all pairs of servers, S k and S h 

Z.-iL ._* a ijjt* u ij xc i 

E n „ g s l 



1 5. A method as recited in claim 4, wherein if the content delivery system 

3 

j2 should add one or more servers, or remove one or more customers, the load of each server wi 

J 

3 not be redistributed unless the maximum allowed deviation from a balanced state 0 is 



exceeded. 



1 6. A method as recited in claim 1, further including the step of adding one 

tf2 or more customers to the content delivery system only if AORT new < (1 + O) x AORT old : 

p 

W3 wherein AORT old and AORT mw are old and new values of AORT defined 

Mi 

0 S . " ! S ■ ^ ! Z ^ ! a u* x u u x °i x ( sdel * + ndel M ) 

4 as AORT old = — J - 

n ^ 8 xr m , 



5 and AORT.„., = 



T, i=l T l j =l H k=l a 'u* xu u x c i x ( sdel * + ndel hk ) 



6 wherein a' i j k is a new mapping resulting from the addition of one or 

7 more customers; and 

8 wherein <t> is an allowable change in AORT for existing clients. 
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1 7. A method as recited in claim 1 , further including the step of using a 

2 linear constraint solver to generate the mapping. 

1 8. A method as recited in claim 1, further including the step of using a non- 

2 linear constraint solver to generate the mapping. 

1 9. A method as recited in claim 1, further including the step of using a 

2 heuristic algorithm to generate the mapping, the heuristic algorithm comprising the step of 

f-r3 assigning large a Uk values to small u u x q x (sdel k +ndel jk ) values to produce a smaller overall 

ft AORT value. 

o 

; - 5 
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1 10. A method as recited in claim 9, the heuristic algorithm comprising the 

2 steps of: 

3 generating a plurality of sorted lists by sorting Q values in increasing 

4 order of c h sorting <C„ G,) pairs in increasing order of u ip sorting S k values in increasing order 

5 of sdel k , and sorting (G jf S k ) pairs in increasing order of ndel jk ; 

6 starting with a top-most, smallest value item in each list, identifying 

7 comparable smallest-value items from the other lists to generate a plurality of <C„ G Jf S k ) triples 

8 equivalent to the number of sorted lists; 

9 selecting from the plurality of <Q Gj,S k ) triples, the <C„ Gj,S k ) triple with 
QjO the smallest u itJ x qx (sdel k +ndel jk ) value; 

hjl assigning to a server S k of the selected (Q, G jf S k ) triple a remaining load 

Hf2 from the <Q G y ) pair; and 

v4 

143 repeating the heuristic algorithm starting with generating the plurality of 

Q4 sorted lists, taking into account the changes in the values of the Q values and the <QG y > pairs 

Hs as a result of the previous server assignment during each iteration, until the load from all 

H6 <C„ Gp pairs has been assigned to a server S k ; 

if""! 

Q7 wherein if, during any iteration of the heuristic algorithm, the load 

18 capacity of the server S k is not sufficient to handle the remaining load, the remaining load 

19 capacity of the server S k is assigned to some of the load of the (Q, G 7 > pair, and an unassigned 

20 portion of the load from the (QGy) pair is reinserted into the iterative process. 
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1 1 1. A method as recited in claim 10, the heuristic algorithm further including 

2 the steps of: 

3 generating a load-capacity prioritized sorted list by sorting C, values in 

4 decreasing order of remaining load capacity s k ; 

5 starting with a top-most, largest value item in the load-capacity 

6 prioritized list, identifying comparable smallest-value items from the other lists to generate a 

7 load-capacity prioritized (Q G jf S k ) triple; 

8 considering the load-capacity prioritized (C if G jr S k ) triple in the selection 

9 of the top-most (C if Gj,S k ) triple with the smallest u W} x qx (sdel k +ndel jk ) value; and 

10 repeating the heuristic algorithm starting with generating the plurality of 

11 sorted lists, taking into account the changes in the values of the C t values, the <QG ; ) pairs, and 
rt2 remaining load capacity as a result of the previous server assignment during each iteration, 
J(3 until the load from all {C if Gp pairs has been assigned to a server 5^. 

O 
yj 

ill 12. A method as recited in claim 10, the heuristic algorithm further including 

!jf2 the steps of: 

s 3 generating a list of content-available (C ir S k ) pairs in which the content of 

M 

Q4 customer C t is stored in server S k \ and 

selecting the <Q G jf S k ) triple with the smallest u u x c, x (sdel k +ndel Jk ) 

W6 value that is also part of the list of content-available (C i9 S k ) pairs; 

7 wherein if, during any iteration of the heuristic algorithm, there is no 

8 (C if G p S k ) triple that is also part of the list of content-available (C ir S k ) pairs, a suitable (C if G jf S k ) 

9 triple with the smallest u^xc^sde^+ndel^j) value is chosen and the data of customer C, is 
10 migrated to server 
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1 13. A method as recited in claim 12, the heuristic algorithm further including 

2 the step of generating a list of content-unavailable (QS*) pairs in increasing order of migration 

3 time penalty for which the content of customer Q is not stored in server S k ; 

4 wherein if, during any iteration of the heuristic algorithm, there is no 

5 (C it G jf S k ) triple that is also part of the list of content-available (C if S k ) pairs, a suitable <C„ Gj,S k ) 

6 triple with the smallest combined u ij xc i x(sdel k +ndel Jf ) value and (C ir S k ) migration time penalty 

7 is chosen and the data of customer C, is migrated to server S k . 

r^l 14. A method as recited in claim 1, further including the step of estimating 

^2 ndel jk for non-persistent connections using HTTP logs, the step of estimating ndel Jk for non- 

W3 persistent connections using HTTP logs comprising the steps of: 

L-Jt 

hj4 computing an estimated round trip delay Ar server cliem as t con _ req rec (i+ 1) - 

f 5 Imp send end(0 ^ om information stored in the HTTP logs, where t con req rec (i + l) represents a time 

Jf6 at which a connection request message is received by the server for an (i+ 1)* object, and 

W 7 t res P _send_enJJ) represents a time at which the server stops sending an i th object; and 

^8 computing the response time as 

™ Ar r t 

9 (t „ .-t ) + 2x " erver,c , where t m rea rec represents a time at which a 

V resp _ send end con _ req _ rec / ^ conjeqjrec r 

10 connection request message is received by the server for an object, and t respjend end represents a 

1 1 time at which the server stops sending the object. 
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1 . 15. A method as recited in claim 1, further including the step of estimating 

2 ndel j k for persistent connections using HTTP logs, the step of estimating ndel jk for persistent 

3 connections using HTTP logs comprising the steps of: 

4 computing an estimated round trip delay Ar servertCliefU as t con close rec - 

5 t resp send end (last), where t con _ close _ rec represents a time at which the server receives a request to 

6 close the persistent connection, and t respjend en JJast) represents a time at which the server stops 

7 sending a response for a last request; and 

8 computing the response time as 

& {t cm _ d05e rec w fon _ re? r J + ^^-^^ . where t conjeqjec represents a connection 

H?0 request time. 

3 : jt 

j^l 16. A method for estimating a per-byte network delay a observed by a 

s 2 requesting entity in a geographic location while retrieving data from a server using TCP logs, 

p3 the method comprising the steps of: 

Ly 

£24 computing an immediate round-trip delay Ar imm between the server and 

Jr?5 the requesting entity as t con est - t ack send from information stored in the TCP logs, where t con est is 

6 a time at which a connection with the server is established, and t ack send is a time at which the 

7 server sends an acknowledgement to a connection request message received from the 

8 requesting entity; and 

9 determining the per-byte network delay a using the computed immediate 

10 round-trip delay Ar /mm , an equation t ack recei Jk) - t data _ se Jl)=Ar imm +a x (k-l), and information 

1 1 stored in the TCP logs, where send (l) is a time at which an I th byte of the data has been sent 

12 by the server, and t ack receive (k) is a time at which acknowledgement for a k h (k>l) byte of the 

13 data is received by the server. 
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1 17. A method for estimating a response time observed by a requesting entity 

2 in a geographic location while retrieving objects from a server through a non-persistent 

3 connection using HTTP logs, the method comprising the steps of: 

4 computing an estimated round trip delay Ar server clierU as t conreqrec (i+l) - 

5 t resp se ^_ e JJ) from information stored in the HTTP logs, where t con _ req rec (i + 1) represents a time 

6 at which a connection request message is received by the server from the requesting entity for 

7 an (i + 1)* object, and t respjend er Jj) represents a time at which the server stops sending an i th 

8 object to the requesting entity; and 

^9 computing the response time as 

Q0 (t resp send end -t con req rec ) + 2 x Ar ^ lient 9 where t con req rec represents a time at which a 
W " 1 

5=^1 connection request message is received by the server from the requesting entity for an object, 

42 and t resp send enA represents a time at which the server stops sending the object to the requesting 

§43 entity. 



- — , 



— 1 18. A method for estimating a response time observed by a requesting entity 

3 2 in a geographic location while retrieving objects from a server through a persistent connection 

3 using HTTP logs, the method comprising the steps of: 

4 computing an estimated round trip delay Ar server client as t con closejec - 

5 t resp send en JJtast), where t con close rec represents a time at which the server receives a request to 

6 close the persistent connection, and t respjend e JJast) represents a time at which the server stops 

7 sending a response for a last request; and 

8 computing the response time as 

9 (t con close rec -t con rev r J ^ rse ^ n ' -^==^, where t con reqjec represents a connection 
10 request time. 
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1 19. In a content delivery network having m servers, S ' = {Sj , . . . ,S m } , n active 

2 customers , C" = { Q , . . . , C n } , and g geographic locations , G ' = { Gj , . . . , G g } , a content delivery 

3 system for distributing server loads, the content delivery system comprising: 

4 memory for storing a server delay sdel k of server S kJ a network delay 

5 ndel j k observed by customers in geographic location G, while retrieving content from server S k , 

6 a priority value p j for customer C h a total load q of customer Q", a fraction of requests u u 

7 coming to customer C t from region G ; , a mapping a ijk representing a fraction of requests 

8 coming to customer C t from region G ; that have been redirected to server S k9 and a load 
p9 capacity s k of server S k ; and 

j4o a processor programmed for 

Rf 1 representing an average prioritized observed response time as 

y 

hi Z ^iZ ^iZjl^i x x c i x A x + ndel M ) 

H2 AORT = —±—± — J — — — ,and 

^ n 

2,. * c i*Pi 

H= z = 1 

yl3 generating a mapping that assigns requests from customers to a 

^44 particular server while minimizing AORT. 

3 

1 20. A system as recited in claim 19, the processor further programmed for 

2 assigning all requests from all customers in all regions to a particular server such that, for each 



3 qeC',G g eG\ £ a,,,, =1.0. 



k=\ 



1 21. A system as recited in claim 19, the processor further programmed for 

2 assigning requests to a particular server while ensuring that the load capacity of each server is 

n g 

3 not exceeded such that, for each for each S k e S', ^ ^ a ijk x u i} x c i < s k . 

1=1 7=1 
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1 22. A system as recited in claim 19, the processor further programmed for 

2 assigning requests to a particular server while balancing the load of each server to within a 

3 maximum allowed deviation from a balanced state 0 such that, for all pairs of servers, S k and 



y y a i u i 

v n ^ g 



S h — — — <(l + 0)x-*-. 

g s t 



1 23. A system as recited in claim 22, wherein if the content delivery system 

2 should add one or more servers, or remove one or more customers, the processor is further 

3 programmed for not redistributing the load of each server unless the maximum allowed 
.4 deviation from a balanced state 0 is exceeded. 
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1 24. A system as recited in claim 19, wherein the processor is programmed 

2 for allowing one or more customers to be added to the content delivery system only if 

3 A0RT new <(l + O)xA0RT 0ld : 

4 wherein AORT old and AORT new are old and new values of AORT defined 

n v S m 



5 as AORT^ = 



Z . = l Z . = 1 Z k = l a i,i,k x u u x c i x ( sde h + nde hM ) 



O ^ n ^ g ^ m r 



M6 and AORT 



m ^i = \ ' 

r!J7 wherein a' i j k is a new mapping resulting from the addition of one or 

s 8 more customers; and 

q9 wherein O is an allowable change in AORT for existing clients. 

lit 

pi 25. A system as recited in claim 19, the processor further programmed for 

w 2 using a linear constraint solver to generate the mapping. 

1 26. A system as recited in claim 19, the processor further programmed for 

2 using a non-linear constraint solver to generate the mapping. 

1 27. A system as recited in claim 19, the processor further programmed for 

2 generating the mapping by assigning large a iJ k values to small u u x c, x (sdel k +ndel j k ) values to 

3 produce a smaller overall AORT value. 
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1 28. A system as recited in claim 27, the processor farther programmed for: 

2 generating a plurality of sorted lists by sorting C, values in increasing 

3 order of c h sorting <Q G y > pairs in increasing order of u ij9 sorting S k values in increasing order 

4 of sdel k , and sorting (G jf S k ) pairs in increasing order of ndel jk \ 

5 starting with a top-most, smallest value item in each list, identifying 

6 comparable smallest- value items from the other lists to generate a plurality of (C if Gj,S k ) triples 

7 equivalent to the number of sorted lists; 

8 selecting from the plurality of <Q G jt S k ) triples, the <Q G Jf S k ) triple with 

O 

y|9 the smallest u Uj x q x (sdel k +ndel jk ) value; 

assigning to a server S k of the selected (C if G jf S K ) triple a remaining load 

|¥l from the <Q, G y > pair; and 

rife repeating the heuristic algorithm starting with generating the plurality of 

J3 sorted lists, taking into account the changes in the values of the Q values and the <Q,G ; ) pairs 

f4^ as a result of the previous server assignment during each iteration, until the load from all 

(C if G^ pairs has been assigned to a server S k ; 
QH5 wherein if, during any iteration of the heuristic algorithm, the load 

17 capacity of the server S k is not sufficient to handle the remaining load, the remaining load 

18 capacity of the server S k is assigned to some of the load of the (C if Gj) pair, and an unassigned 

19 portion of the load from the (C„ Gj) pair is reinserted into the iterative process. 
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1 29. A system as recited in claim 28, the processor further programmed for: 

2 generating a load-capacity prioritized sorted list by sorting C, values in 

3 decreasing order of remaining load capacity s k \ 

4 starting with a top-most, largest value item in the load-capacity 

5 prioritized list, identifying comparable smallest-value items from the other lists to generate a 

6 load-capacity prioritized (C if G jf S k ) triple; 

7 considering the load-capacity prioritized (C ir G jr S k ) triple in the selection 
_8 of the top-most (C if G jt S k ) triple with the smallest u u x c, x (sdel k +ndel Jk ) value; and 

S9 repeating the heuristic algorithm starting with generating the plurality of 

g|0 sorted lists, taking into account the changes in the values of the C ( values, the <C,,G y > pairs, and 

2l remaining load capacity as a result of the previous server assignment during each iteration, 

\*4l until the load from all (C if G.) pairs has been assigned to a server S k . 

i j. 

JI 1 30. A system as recited in claim 28, the processor further programmed for: 

generating a list of content-available (C if S k ) pairs in which the content of 

03 customer C, is stored in server S k \ and 

"4 selecting the (Q G p S k ) triple with the smallest u u x q x (sdel k +ndel j k ) 

5 value that is also part of the list of content-available (C if S k ) pairs; 

6 wherein if, during any iteration of the heuristic algorithm, there is no 

7 (C if Gj,S k ) triple that is also part of the list of content-available (C if S k ) pairs, a suitable (C if G jf S k ) 

8 triple with the smallest u^xc^isdel^ndelj ^ value is chosen and the data of customer Q is 

9 migrated to server S k . 
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1 31. A system as recited in claim 30, the processor further programmed for 

2 generating a list of content-unavailable (C if S k ) pairs in increasing order of migration time 

3 penalty for which the content of customer C, is not stored in server S k ; 

4 wherein if, during any iteration of the heuristic algorithm, there is no 



5 (C if Gj,S k ) triple that is also part of the list of content-available (C if S k ) pairs, the processor is 

6 further programmed for selecting a suitable (C if G Jf S k ) triple with the smallest combined 

7 u ij xc i x(sdel k +ndel Jk ) value and (C i} S k ) migration time penalty, and migrating the data of 
08 customer Q to server S k . 



Hi 32. A system as recited in claim 19, the processor further programmed for 

M=2 estimating ndel jk for non-persistent connections using HTTP logs by: 

LI 3 computing an estimated round trip delay Ar server cUem as t con reg rec (i + 1) - 

L4 t resp send end (i) from information stored in the HTTP logs, where t con req _ rec (i+ 1) represents a time 

O5 at which a connection request message is received by the server for an (/+ 1)* object, and 

M 6 t resp send en Jii) represents a time at which the server stops sending an /'* object; and 

S 7 computing the response time as 

8 (t resp _ send _ end - t con req _ rec ) + 2 x Ar ^ lient , where t conjeqjec represents a time at which a 



9 connection request message is received by the server for an object, and t resp send end represents a 
10 time at which the server stops sending the object. 
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1 33. A system as recited in claim 19, the processor further programmed for 

2 estimating ndel jk for persistent connections using HTTP logs by: 

3 computing an estimated round trip delay Ar serverfCliera as t con close rec - 

4 t resp send eJJiast), where t con _ closejec represents a time at which the server receives a request to 

5 close the persistent connection, and t resp send er Jlast) represents a time at which the server stops 

6 sending a response for a last request; and 

7 computing the response time as 

|8 {t concloserec -t con ^_ r ^ ^ where t„ _ reqjec represents a connection 

q9 request time. 
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